Night Vision Camera Fusion with Natural Colors Using a Spectral / Texture Based Material Identification Algorithm

Combining multiple types of night vision cameras, such as low-light-level visible to near infrared (VNIR), short-wave (SWIR), mid-wave (MWIR), and/or long-wave infrared (LWIR), enables the presentation of complimentary information into a single display. A great deal of developmental work has been devoted to maximizing the information in the fused imagery; however, fusion schemes that rely on some form of mapping or transformation from the multi-band imagery to a composite false-color image do not consistently match recognizable "true" colors and also suffer from color consistency problems as the environment and materials in the scene change. Other related techniques that have proven useful for mapping single band grayscale values to a pseudo-color palette have limited utility for multi-band infrared systems, particularly in dynamic environments where the distribution of brightness values can change. This paper presents a method for combining night vision imagery from multiple cameras into a single fused output with recognizable colors. The colors are determined through a material identification algorithm that relies on a spectral/texture analysis along with additional ancillary information. The calculations are done in a probabilistic manner using a knowledge-based inference technique that considers material database matching together with expected scene content, available via future Global Information Grids and Geographical Information Grid Services. The result is that identified materials can be rendered in natural recognizable colors (e.g., grass is green), which aides in scene understanding, object identification, and also reduces viewer fatigue.